Parameter Estimation and Filtering Using Sparse Modeling

Sparsity-based estimation techniques deal with the problem of retrieving a data vector from an undercomplete set of linear observations, when the data vector is known to have few nonzero elements with unknown positions. It is also known as the atomic decomposition problem, and has been carefully studied in the field of compressed sensing. Recent findings have led to a method called basis pursuit, also known as Least Absolute Shrinkage and Selection Operator (LASSO), as a numerically reliable sparsity-based approach. Although the atomic decomposition problem is generally NPhard, it has been shown that basis pursuit may provide exact solutions under certain assumptions. This has led to an extensive study of signals with sparse representation in different domains, providing a new general insight into signal processing. This thesis further investigates the role of sparsity-based techniques, especially basis pursuit, for solving parameter estimation problems. The relation between atomic decomposition and parameter estimation problems under a so-called separable model has also led to the application of basis pursuit to these problems. Although simulation results suggest a desirable trend in the behavior of parameter estimation by basis pursuit, a satisfactory analysis is still missing. The analysis of basis pursuit has been found difficult for several reasons, also related to its implementation. The role of the regularization parameter and discretization are common issues. Moreover, the analysis of estimates with a variable order, in this case, is not reducible to multiple fixed-order analysis. In addition to implementation and analysis, the Bayesian aspects of basis pursuit and combining prior information have not been thoroughly discussed in the context of parameter estimation. In the research presented in this thesis, we provide methods to overcome the above difficulties in implementing basis pursuit for parameter estimation. In particular, the regularization parameter selection problem and the so-called off-grid effect is addressed. We develop numerically stable algorithms to avoid discretization and study homotopy-based solutions for complex-valued problems. We use our continuous estimation algorithm, as a framework to analyze the basis pursuit. Moreover, we introduce finite set based mathematical tools to perform the analysis. Finally, we study the Bayesian aspects of basis pursuit. In particular, we introduce and study a recursive Bayesian filter for tracking the sparsity pattern in a variable